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ABSTRACT 


The purpose of this study was to develop and evaluate a set of icons for the 
next generation message processing system for the TACAMO airborne strategic 
communications platform. An icon set for a proposed interface was developed 
through the use of an icon production method test, that is, potential users designed 
candidate icons that were meaningful to them. These icons were then refined for 
discriminability via input from a user survey. To determine if well-developed icons 
with alphanumeric labels yield a significant performance advantage over the same 
icons without labels, an experiment involving trained users was conducted using a 
response time model. Subtractive logic was used to measure icon identification times 
as a function of whether they were or were not labeled. When speed of performance 
and rate of errors were compared, labeling of icons resulted in significantly longer 
response times, yet did not result in fewer errors for the tested icon set. It is 
recommended that the unlabeled set of icons be used for TACAMO’s next 
generation message processing system, and that the icon production method be used 


more widely to involve users in interface design. 
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I. INTRODUCTION 


A. TACAMO STRATEGIC COMMUNICATIONS 

With the weight of the United States strategic nuclear triad shifted firmly to the 
Navy's ballistic missile submarine (SSBN) fleet (Schmitt, 1991, and Talbott, 1991), 
the TACAMO’ communications system has assumed a vital role in the world of 
nuclear deterrence. TTACAMO’s primary mission is to provide the National 
Command Authorities (NCA) a survivable and endurable means to command the 
nation’s strategic fleet of ballistic missile submarines. Despite a rapidly changing 
world, at the core of all long-range defense planning remains the need to minimize 
the likelihood of the extreme case—the all-out nuclear attack. Deterrence against 
any attack requires the assured survival not only of powerful retaliatory forces, but 
also of the command and control system that controls them (Iklé and Wohlstetter, 
1988, p. 35). 

Conceived during the height of cold war tensions, the —TACAMO 
communications system was originally envisioned as a temporary fix until a hardened 
shore-based system could be developed to communicate with the ballistic missile 


submarine fleet. But, as the accuracy of Soviet missiles improved, it became 


"TACAMO, a term used interchangeably for a specific TACAMO aircrewor the entire TACAMO 
communication system, is derived from the command "take charge and move out." In 1963, the 
Director of Naval Communications, then Rear Admiral Bernard F. Roeder, U.S. Navy, coined the 
phrase at the first meeting of the then highly-classified project to determine if an airborne Very Low 
Frequency (VLF) communications system was feasible. 


apparent that any fixed geographical locations would be vulnerable. In September 
1973, TACAMO assumed the key role of providing a continuously airborne and 
survivable method of relaying retaliatory orders to deployed ballistic missile 
submarines in the event of a nuclear attack. (Sanders, 1991, p. 175) 

TACAMO is a part of the Worldwide Airborne Command Post System 
(WWABNCP) which provides alternative command and control. The WWABNCP 
ensures that the NCA will retain connectivity with U.S. strategic forces in the event 
of a nuclear attack. TACAMO receives uplinked encrypted and unencrypted 


message traffic over the spread of the radio frequency spectrum (see Figure 1.1). 


TACAMO Communications 


UHF SATCOM Antenna Pods —— 


@. DOP ” 


VLF Trailing Wires 


\ 
VLF Receive Antenna 


ee 


= HF Subsystem 





Figure 1.1. TACAMO Airborne Communications Platform 


Messages are sorted and prioritized for routing by the Airborne Communications 
Officer(ACO) and communications crew aboard the aircraft. The TACAMO 
Message Processing System (TMPS) is used for automated message storage and 
forwarding to the aircraft’s very low frequency (VLF) transmitter. 

The next TACAMO communications upgrade, scheduled for the mid-1990s, 
entails the modification of the E-6A aircraft to include a Military Strategic and 
Tactical Relay (MILSTAR) system with the associated MILSTAR Message 
Processing System (MMPS), which is an upgrade of the older TMPS. MILSTAR is 
a joint development program with the Air Force which will incorporate anti-jam 
extremely-high frequency capabilities, providing the Navy with additional spread- 
spectrum survivable satellite communications. 

Federman (1988) has described human factors guidelines for developing 
software for the TMPS upgrade and recommended further experimental studies in 
designing display formats for the TMPS. Budget restrictions have limited further 
studies; current plans are to adapt the existing command-line processor of the TMPS 
into an operating system for the MMPS with little or no alteration. The study 
reported here has integrated Federman’s recommendations into a graphical interface 
design and specifically developed and evaluated a set of icons for the TACAMO 


MMPS. 


B. GRAPHICAL INTERFACE FOR MILITARY COMPUTER APPLICATIONS 


1. Performance Criteria 

Computer processing power and high-resolution graphics for snl 
computer applications have improved dramatically during the past decade. Software 
designers and developers are turning to graphical interfaces and display formats 
including icons, because pictorial symbols can generally be recognized more rapidly 
and accurately than words (Green and Pew, 1978, p. 103). Haber (1970), for 
example, suggests that it is easy for humans to recall pictorial stimuli and that the 
capacity of memory for pictures may even be unlimited. Icons offer established 
benefits when developed with creative design methods, because they reduce 
memorization of commands and syntax (Brown, 1986, p. 87) and produce quicker 
and easier recall from memory when compared to nonpictorial symbols (Florence 
and Geiselman, 1986, p. 404). It makes sense, therefore, to use selectively screened 
icons to facilitate performance within graphical interfaces. 

Replacing the vague and misleading notion of "user friendliness" with 
measurable human factors performance criteria should be a top priority of human- 
computer interface design. Shneiderman (1987) proposes that debates over user 
friendliness can be avoided by using five quantitative acceptance criteria: 

1. Subjective acceptance by user. 
2. Initial training time. 


3. Retention of skills over time. 


4. Speed of performance. 

5. Rate of errors by the user. 
Once a decision about the relative importance of each of these human factors 
criteria has been made, specific acceptance tests should be established to guide 
designers and inform prospective users (Shneiderman, 1987, pp. 396-397). While 
icons are only one of many interface methods that can be employed, they offer great 


potential for use in military software applications. 


2. Designing the Optimal Icon 
While the design of icons cannot be summarized with a few rules, 
Lodding (1983, p. 19) offers guidelines that can be used to characterize an optimal 
Icon: 
1. When first encountering the icon, a viewer should be able to infer its intended 
meaning. Additionally, once learned, the meaning should be perceived as 


appropriate. 


2. When selecting an icon from a menu or icon set, only one icon should appear 
aS appropriate. 


3. The icon should not have any unnecessary ambiguous connotations. 

Clarity, consistency, familiarity, and simplicity are the key attributes of good 
icon design (Marcus, 1991). Yet rational analysis and good graphic design practice 
alone cannot guarantee an optimal icon. The targeted user must be included in the 
development process to establish the suitability and interpretability of the design 


(Cahill, 1975, p. 380). 


3. Icons: The Label Debate 

While it 1s clear that a careful and creative design process is required to 
develop a successful set of icons, the literature indicates a lack of consensus 
regarding the use of icons, alphanumeric formats, and labels. While Steiner and 
Camacho (1989, p. 14) argue that, for large amounts of information, users perform 
better with icons than with alphanumeric formats, Ells and Dewar (1979, p. 167) 
found that for small amounts of information users perform just as well with 
alphanumeric formats as with icons, and sometimes perform even better. 

Kantowitz (1985) notes that redundancy, such as that supplied by labels, 
tends to slow down a system somewhat while raising its reliability; however, 
redundant information perceived through two sensory channels simultaneously (e.g. 
the eyes and ears) actually speeds up processing time, compared to cases where 
information is perceived through a single channel. In contrast, Pellegrino, Siegel, 
and Dhawan (1975) warn against the supplementary use of labels, using the dual- 
coding hypothesis that pictures are encoded both visually and acoustically, while 
labels are primarily acoustically encoded and thus may distract from pictures. Yet 
another view is presented by Guastello, Traut, and Korienek (1989, pp. 118-119), 
who found that mixed modality icons, consisting of pictorial icons with alphanumeric 
labels, are rated as more meaningful than icons that consist of verbal or pictorial 
elements only. 

MIL-STD-1472D, Human Engineering Design Criteria for Military Systems, 


Equipment and Facilities (1989) requires the use of labels for icons. Paragraph 


5.15.4.8.3 of that document states, "Where icons are used to represent control actions 
in menus, verbal labels shall be displayed with each icon to help assure that its 
intended meaning will be understood." The varied results reported in the literature 
suggest that the strict labeling requirements of MIJL-STD-1472D may be imposing 
unrealistic requirements on military interface designers. Experiments on the effect 
of icon design format in military applications are urgently needed. With the 
controversy surrounding speed of recognition and rate of errors by the user, studies 


should focus on analyses based on Shneiderman’s last two acceptance criteria. 


C. GOALS AND OBJECTIVES 

The goal of this study is to determine if well-developed icons with alphanumeric 
labels yield a significant performance advantage over the same icons without labels. 
This study has concentrated on a set of icons proposed for a computer display format 
for TACAMO’s anticipated message processing system upgrade. Several objectives 
have been met to achieve the primary goal. 

1. Screen Layout and Tasks. A prototype display format for the MMPS system 
was designed, a set of screen design guidelines was documented, and a set of 
tasks to be represented on the screen by icons was determined. 

2. Initial Icon Design. TACAMO aircrew personnel were used to sketch symbols 
and to suggest labels for the TACAMO MMPS task descriptions. The resulting 
set of icons was refined to enhance discriminability. 

3. Icon Identification Survey. Recognition and confusability of the proposed icons 


were determined by surveying TACAMO aircrew personnel, to obtain a final 
set of icons for evaluation. 


4. Icon Design Evaluation. Two icon design formats, with and without labels, 
were tested to measure speed and accuracy of identification under the two 
conditions. 
D. SCOPE : 

The scope of this study is limited a single set of icons developed specifically for 
use with the TACAMO MMPS. Speed and accuracy of recognition were measured 
for the icons alone and for icons that were labeled, to determine whether a 
significant performance difference could be noted. Tests were conducted in a 
controlled environment without stress or the requirement to carry out supplementary 
tasks. It is not the intent of this study to validate the use of icons in general for 
military computer applications. The goal is to present a method of developing icons, 
to evaluate the resulting icons, and to make recommendations that may be used 


during the development of the TACAMO MMPS. 


Il. INTERFACE DEVELOPMENT AND INITIAL ICON DESIGN 


A. TACAMO HUMAN FACTORS ANALYSIS 

Several human factors analyses have been carried out that are directly 
applicable to the TACAMO systems and especially to the TACAMO Message 
Processing System (TMPS). An analysis of the TACAMO Communications Central 
conducted by Dean and Schlumbrecht (1981) details a typical TACAMO mission 
scenario, including descriptions of the tasks required for operating Communications 
Central. Tasks related to the preflight, mission assumption, mission relief, landing, 
and post landing phases are described in the report. 

Dean and Mitchell’s (1981) study outlines human factors guidelines for the 
TACAMO airframe replacement project. General guidelines centered on reducing 
crew Station work load by designing Communications Central operator consoles to 
minimize stress and compensate for types of errors that are most likely to occur 
(Dean and Mitchell, 1981, p. 31). Moreover, Dean and Mitchell note that sleep loss 
and rest disruption are among the most severe stresses on TACAMO crew members, 
and many of the tasks most vulnerable to sleep-loss effects are centered on the 
operation of the TMPS. These tasks include the following: 

1. Monitoring tasks (such as checking radio circuits). 


2. ‘Tasks that are new or require learning on the job (e.g., training of new TMPS 
operators). 


3. High-workload tasks that require time-sharing with other primary and 
secondary tasks (e.g., receiving and transmitting Emergency Action Messages). 


4. Tasks that require continuous attention and steady performance (e.g., 
monitoring the TMPS and the overall communications central system). 


Given the in-flight refueling capability of the E-6A, much longer TACAMO 
missions than those currently flown are being envisioned. The anticipated resultant 
increase in sleep loss highlights the importance of crew rest procedures and of 
careful mission planning for coordinated work-rest cycles. The established need for 
an upgrade to the TMPS has reached a new level of urgency with foreseeable 
TACAMO mission lengths of up to 72 hours (Sanders, 1991, p. 179). 

Federman (1988) makes specific recommendations to correct existing TMPS 
deficiencies, based on a field survey conducted at the VQ-4 squadron, Naval Air 
Station, Patuxent River, Maryland. Primary areas of concern are data entry, data 
display, interaction control, feedback, prompts, error management, and data 
protection. Several factors to consider in satisfying a "well human factored screen" 
are outlined by Federman (1988, p. 11): 

1. Reduction of memory load. 

2. Consistency in language and appearance on the screen. 

3. Orderly, clean, and clutter-free screens. 

4. Understandability of abbreviations, acronyms, and natural language. 


5. Use of headings, captions, instructions, and options for improved 
meaningfulness. 


6. A simple way of retrieving information that is stored in the system. 
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7. Display of the "right amount of information" on the screen without overloading 
it. 


8. Help for the user when difficulty arises. 


B. MMPS PROTOTYPE SCREEN LAYOUT AND MENU DESIGN 

Federman’s screen characteristics form a sound basis for designing a human- 
computer interface that is well accepted by TACAMO users. With the human 
factors deficiencies of the current TMPS thoroughly documented (Federman 1988), 
careful consideration and planning should go into the development of software for 
the next generation MILSTAR Message Processing System (MMPS). 

The TACAMO MMPS, a dual Rolm Hawk/32 computer with increased 
memory storage, is scheduled to replace the current TMPS after 1995 (Sanders, 1991, 
p. 181). Current plans are to modify the existing TMPS operating system software 
into an Ada-compiled version for the MMPS until funding can be secured for new 
software. An initial prototype design for the TACAMO MMPS interface has been 
developed, incorporating the design recommendations of Federman and other 
TACAMO Airborne Communications Officers (ACOs) stationed at the Naval 
Postgraduate School. 

The TACAMO MMPS prototype is designed to encompass many of the 
functions currently accomplished with separate pieces of equipment in 
Communications Central. The majority of equipment function settings can be 
controlled at an MMPS terminal with the addition of an enhanced data bus. Design 


guidelines have centered on creating an interface that includes not only current 


11 


TMPS functions, but functions that are currently accomplished at other crew 
positions in Communications Central. Communications Central ideally will have two 
MMPS terminals, a primary terminal and a secondary terminal serving as a training 
center, document database, and operational backup. This second terminal would 
permit conversion of aircraft maintenance manuals and operational documents into 
CD-ROM format, saving significant weight and storage space that could be better 
used for survival supplies and aircraft spare parts. 

Based on the human factors analyses discussed earlier, recommendations have 
been documented for the TACAMO MMPS prototype screen layout and menu 
design. These recommendations are included in Appendix B. In anticipation of 
increased color and graphics capabilities, a graphical human-computer interface 
was developed for the MMPS prototype. Appendix B.1 outlines guidelines for the 
MMPS interface, including design principles, general display characteristics, 
interaction characteristics, information presentation formats, and user assistance. 


Menu structure recommendations are outlined in Appendix B.2. 


C. PICTORIAL SYMBOLS AND INITIAL MMPS ICON DESIGNS 

Over 300 years ago, the German philosopher and mathematician Baron 
Gottfried Wilhelm Von Leibniz (1646-1716) planted the seeds of symbolic logic and 
computer design (Kreiling, 1968, p. 100). Leibniz’ dream was that someday a 
universal system of pictorial symbols would exist that could be read in all languages 


without having to be translated. A symbol can give an identity to a subject and, by 
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repeated use, can come to equal it (Holmes, 1990, p. 11). By presenting user 
commands and system information in the form of pictorial symbols (often referred 
to as icons), graphic displays reduce the time and effort of learning, facilitate user 
performance, and reduce errors (Lodding, 1983, p. 11). 

Most symbols have meaning only within a given context. The context in which 
the icons developed here are meaningful is that of the MMPS terminal and the tasks 
performed there. Appendix B.3 lists 20 tasks that will be carried out on an MMPS; 
these were selected for initial icon representation. The task descriptions were 
developed from thumbnail sketches provided by TACAMO ACOs at the Naval 
Postgraduate School (NPS). 

Three approaches to design of pictorial symbols were considered in developing 
the MMPS task icons. The first approach considered assembling general ideas of 
what a group of ACOs think task icons should look like. The second approach 
considered consulting the Symbol Sourcebook (Dreyfuss, 1972) and Handbook of 
Pictorial Symbols (Modley, 1976) to identify icons for these tasks based on symbols 
developed by industrial designers for various purposes. Neither of these methods, 
however, qualifies as a scientific approach to the development of a satisfactory set 
of icons specific for the MMPS. The third approach considered is referred to as the 
population-stereotype production method (Mudd and Karsh, 1961; Howell and 
Fuchs, 1968; Green, 1979). This was the primary approach selected for MMPS icon 
development, with some modifications, to generate a set of icons that could be used 


for further study. 
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Iii. ICON PRODUCTION METHOD TEST 


A. BACKGROUND 
The production method technique solicits users of an unbiased subject 
population to draw symbols for specific concepts. Karsh and Mudd (1962) compared 
the accuracy of identification of vehicle control symbols developed by designers with 
and without data from the production method technique. Their results showed that 
the production-aided symbols were significantly superior to symbols developed 
without the production method, indicating that the "design and effectiveness and 
ultimate effectiveness of any symbol is entirely dependent upon the prevailing 
concepts that the specific user population may have of its existing equipment.” 
The production method for designing icons is an effective, low-cost technique 
for generating new symbols representing software tasks in a graphical format. There 
are four essential steps in the icon production method: 
1. Determine concepts for software tasks from the user population. 
2. Identify the subject population that will draw icons. 
3. Elicit icons from the subjects. 
4. Summarize and analyze the data. 
Candidate icons are then developed from categories suggested by the subjects’ 
drawings. Green (1979, p.77) notes that subjects’ drawings are strongly influenced 


by their limitations as artists and their responses should be viewed as suggestive and 
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not definitive. A thorough qualitative analysis of the test data combined with human 


factors design strategies will yield a superior set of candidate icons. 


B. METHOD 


1. Questionnaire Development 
The TACAMO Icon Production Questionnaire presented in Appendix C.1 
was developed to gather the data, using guidelines outlined by Tull and Albaum 
(1973). The questionnaire had three parts. The first part requested biographical 
data. The second section solicited respondents’ opinions on menu structure 
development, not directly related to icon production. The final section consisted of 
one-inch square boxes in which the participants were asked to sketch proposed 


drawings and write out suggested labels. 


2. Participants 

A sample of 12 TACAMO aircrew personnel was drawn from the VQ-4 
squadron, Naval Air Station, Patuxent River, Maryland. Ten participants completed 
the test, while two aircrewmen were unable to complete the testing due to an 
unscheduled TACAMO alert launch. All participants were Airborne 
Communications Officers (n=6) or enlisted Airborne Communications Supervisors 
(n=6) and ranged in age from 22 to 36. Various levels of TACAMO experience 
were reflected in the sample, with a mean of 2.5 years of experience and standard 
deviation of 1.3 years. While the sample of participants was not completely random 


due to the operational constraints and scheduling of aircrew personnel, the 
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participants were deemed to be a reasonable cross-section of TACAMO aircrew 


personnel. 


3. Procedure : 
Participants were assembled as a group for testing. An introduction to 

the purpose and goals of the study was presented verbally by the proctor (Appendix 
C.2). TACAMO Icon Production Questionnaires were then distributed and 
participants supplied biographical data and opinions on menu structure development. 
MMPS task descriptions, randomly-ordered, were then presented verbally and on an 
overhead projector one at a time. Participants were asked to provide a drawing and 


alphanumeric label in separate boxes for each task. —Two minutes were allotted for 


each icon task. Total time required for the test was 50 minutes. 


C. RESULTS 

After the test was administered, the respondents’ drawings and labels were cut 
into squares and assembled by task description. Reduced versions of the 
participants’ drawings appear in Appendix C.3. A list of the proposed alphanumeric 
labels is contained in Appendix C.4. A listing of all of the picture elements that 
participants used to represent task descriptions is included as Appendix C.5. Where 
an objective appraisal of the free-response drawings might be seen as a difficult task, 
respondents were encouraged to explain their drawings with short notes if they felt 


that their drawings might be misinterpreted. 
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The drawings were created without much difficulty for most of the task 
descriptions. A majority of the respondents reported that the test was challenging 
and interesting. Review of the drawings, labels, and picture elements indicates that 
the participants were strongly influenced by the wording of the task description. 
Despite a limited level of artistic experience, respondents developed a variety of 
original ideas for drawings, presumably based on their varying levels of TACAMO 
experience. With the success of icon development largely contingent on the 
development of initial ideas, the production method was an important step in 


generating candidate icons for further analysis. 
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IV. ICON CANDIDATES 


A. OVERVIEW OF ICON DEVELOPMENT 

Two or three versions of each of the 20 candidate icons (see Appendix D.1) 
were developed using the sketches obtained from TACAMO aircrew personnel 
during the icon production method test described in Chapter III. The original 
sketches were revised to improve the general quality of the icons, while retaining the 
intended users’ suggestions. Based on a compendium of guidelines for the design of 
icon-based interfaces (Gittins, 1986), the various versions of each icon were then 
studied and refined until one final candidate icon was prepared for each concept, 
based on icon production method data, improved legibility, and a discriminability 
analysis. The refined set of 20 icon candidates then was presented to an additional 
group of FACAMO aircrew personnel for evaluation, using standard survey 
procedures. The methodology used for development of the candidate set of icons 


is described in greater detail below. 


B. REFINEMENT 

Legibility, the perceptual quality of an icon’s structural features, is central to 
an icon’s capability to convey meaning (Webb, Sorenson, and Lyons, 1989). The goal 
of the icon identification survey was to test whether TACAMO aircrew personnel 


could interpret the meanings of the candidate MMPS icons. Refinements were made 
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to the original TACAMO icon sketches to make identification and interpretation 
easier. 

While the literature on icon refinement is limited, the research on symbol 
refinement techniques and effect on identification is more extensive. For example, 
Green (1979) recommends enhancing pictographic symbol discriminability by 
exploring length-to-width ratio, frontal plane orientation, symbol element repetition, 
strokewidth, and the extent to which enclosed areas are filled in. The results of 
Remington and Williams (1986) suggest that a symbol set is in some respects like a 
list that must be learned, indicating that intra-set similarity plays an important role 
in improving performance or decreasing response times. However, training and 
retention of icons is not anticipated as a problem since Hawkins, Reising, Woodson, 
and Bertling (1984, p. 121) have concluded that even if symbols are not intuitive at 
first glance, they can become easily recognizable after a brief learning period and are 
generally robust to changes in complexity and polarity. 

This study chose to concentrate on the findings of Green and Pew (1978) and 
Green (1979) who suggest that, in developing pictographic symbols such as icons, 
designers should concentrate on enhancing legibility and discriminability to raise 
recognition and lower confusion. High recognition and low confusion lead to 
meaningfulness, a forerunner to memorability. Memorability of the icons is in turn 
a precursor to efficient operation of the system (Guastello and Traut, 1989, p. 119). 

Gittins (1986) provides an excellent summary of available icon design guidelines 


drawn from a variety of fields, including human factors and ergonomics (Easterby, 


ibe, 


1970; Caron, Jamieson, and Dewar, 1980; Shneiderman, 1980), graphic arts 
(Dreyfuss, 1972; Marcus 1982), and computer science (Carroll and Thomas, 1982; 
Lodding, 1982). In summarizing these sources, he discusses the strength of 
metaphors, graphic design alternatives, and icon implementation factors. . 
Cautioning against the overuse of graphical sophistication, Gittins (1986, p. 538) 
notes that "considerable psychological and human factors evidence gained from 
studies of pictographic symbols and signs suggests that simpler icon designs are as, 
or more, usable than complex ones." Appendix D.2, which details the actual 
refinements made to each set of original icon sketches, provides an overview of the 


design techniques used on the TACAMO icons. The final set of candidate icons 


used in the test is shown in Figure 4.1. 


C. DISCRIMINABILITY ANALYSIS 

To provide a performance-based criterion for selecting icons, the final set of 
candidate icons was subjected to the discriminability-index formula developed by 
Geiselman, Landee, and Christen (1982). The discriminability-index formula is based 
on the conclusion that symbols are judged more or less similar on the basis of the 
number of shared versus unique configural attributes, as opposed to primitive 
attributes (number of lines, arcs, etc.). The indices obtained from the formula 


provide an objective predictor in reducing search times for a specific symbol set. 
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Figure 4.1. TACAMO Candidate Icons 


The number of common attributes, unique attributes of each icon, and unique 


attributes of the sample icon domain were tallied as follows: 


Number of where x is the number of the 20 icons 
C = common = x, in the sample icon domain having the 
attributes configural attribute 1. 
Number of where y is the number of the 20 icons 
U = unique = » jy; in the sample icon domain not having 
attributes the configural attribute 1. 
Number of where z is the number of common attributes. 
S = unique = 25 - z, (There were 25 instances of attributes in 
attributes of in total making up the 20 symbols in the 
sample domain sample domain). 
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Geiselman, Landee, and Christen (1982) used standardized regression weights from 

their multiple regression analysis to develop their discriminability-index (D) formula: 
D, = 0.07 [UL4S, |= sear ee. (4.1) 
where the subscript c corresponds to a specific icon. 

While Geiselman, Landee, and Christen (1982) developed their discriminability- 
index formula for specific use with graphic-display symbology, their research provided 
a logical precursor for predicting search times for icons used in human-computer 
interfaces. For example, the candidate icon for Transmit has two configural 
attributes, an aircraft and an arrow (refer back to Figure 4.1). The aircraft is unique 
and the arrow is held in common with four of the 20 icons in the candidate icon set 
shown in Figure 1. Thus from equation (4.1): 

DI = {0.07[(20-1) + (20-4) + (25-5)]} - {0.31[5]} = 2.30. 
Discriminability indices calculated for the original 20 candidate icons are provided 
in Appendix D.3 along the revised discriminability indices calculated when the icon 


set was reduced to 18 icons (as discussed later). 


D. ICON IDENTIFICATION SURVEY 
1. Questionnaire Development 
A TACAMO Icon Identification Survey, contained in Appendix D.4, was 
developed to gather recognition and confusion data on the candidate icons using 
guidelines outlined by Tull and Albaum (1973) and Green and Pew (1978). The 


survey consisted of three sections. The first section explained the background and 
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purpose of the survey with a brief description of the respondents’ tasks. Instructions 
were outlined and biographical data was requested in the second section. The final 
section consisted of 22 task descriptions for matching 20 1-inch square candidate 


icons. 


2. Respondents 
Surveys were distributed to 60 TACAMO aircrew personnel at the VQ-4 
squadron, Naval Air Station, Patuxent River, Maryland, and among TACAMO 
ACOs at the Naval Postgraduate School. TACAMO aircrewmen who participated 
in the icon production method test were ineligible to participate in the survey. 
Respondents included Airborne Communications Officers (n=15) or enlisted 
Airborne Communications Supervisors (n=6) and ranged in age from 25 to 34. 
Various levels of TACAMO experience were reflected in the sample, with a mean 
of 3.5 years of experience and standard deviation of 0.9 years. 
3. Task 
Survey participants were asked to study each icon and select the one best 
meaning from the list of task descriptions. Letter codes corresponding to the task 
descriptions were written underneath each candidate icon. Estimated time to 


complete a survey was 20 minutes. 


E. RESULTS 
Results of the icon recognition survey were favorable, with icon recognition 


rates high and confusion between icons low (see Appendix D.5). Icon recognition 
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rates averaged 94.5%, far exceeding the minimum 75% recognition rate 
recommended by Heard (1974). Only two combinations of icons fell outside Heard’s 
maximum 5% confusion criteria: the Retrieve a Message and Backup a Message icons 
were confused 10% of the time and the Status of Aircraft and Help icons woe 
confused 14% of the time. 

The icon recognition survey verified that candidate icons had been successfully 
refined. The Calendar and Bases icons were dropped from the study after additional 
input from TACAMO ACOs at NPS resulted in adjusted task priorities, with a 


definitive set of 18 TACAMO icons left for final testing (see Figure 4.2). 
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Figure 4.2. TACAMO Icons for Final Testing 
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V. ICON DESIGN FORMAT EXPERIMENT 


A. BACKGROUND 

While icons that satisfied discriminability criteria could have been arbitrarily 
selected from acceptable candidates, an empirical validation of symbol effectiveness 
is highly desirable (Cahill, 1975, p. 379). Of specific interest for this study are the 
speed of performance and rate of errors by the user for icons with and without 
alphanumeric labels. The labels for the TACAMO icon set were generated in a 
manner similar to the way the initial icon sketches were provided. During the icon 
production method test, participants were asked to provide alphanumeric labels to 
represent task descriptions (see Appendix C.4). 

To meet minimum legibility requirements prescribed by Siebert, Kosten, and 
Potter (1959), Baker and Nicholson (1967), and Vartebedian (1971), a 15-point WP 
Courier Simplex font from WordPerfect Corporation’s DrawPerfect™ presentation 
graphics software was used to label the icons on a standard VGA 640 480-pixel 
display monitor. The resulting label was 10/16-inch high, or approximately 10 to 12 
scan lines per character height. The TACAMO label suggestions were revised 
slightly in order to meet size and font restrictions and can be seen in Figure 5.1 
(note that the legibility of the printed labels does not necessarily correspond to the 


legibility of the same labels on a graphics display). 
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Figure 5.1. Labeled TACAMO Icons for Final Testing 


B. EXPERIMENTAL DESIGN 

The design used for this experiment was a one-way between-subjects 
comparison of labeled and unlabeled icon stimuli (see Figures 4.2 and 5.1). A 
between-subjects design was chosen to avoid confounding with a training bias due to 
learning the same icon set twice. All subjects received an 18-icon training sequence, 
review training, baseline reaction testing, additional review training, and icon 
recognition testing. Presentation order was completely randomized along with icon 


position in the 3X3-icon matrices. 
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1. Reaction Time Model 
The selection of skill-based actions generally has been studied by 
measuring reaction time (Wickens, 1992, p. 313). The time to react in a situation in 
which any one of several signals may occur must include time to complete four 
processes (Welford, 1980). 


1. Reception of the signal by a sense organ and conveyance of data by afferent 
nerves to the brain. 


2. Identification of the signal. 
3. Choice of the corresponding response. 
4. Initiation of the action that constitutes the response. 

Card, Moran, and Newell (1982) model the human as an "information- 
processor’ through a specific, simple model of perceptual, cognitive, and motor 
processes. The Keystroke-Level Model developed by these researchers defines the 
time to complete a unit task as the sum of acquisition time and response or 
execution time: 


= T 


acquire 


ell 


execule ° 


T 


unit task 


(5.1) 
For the icon design format experiment, acquisition time for a unit task depends on 
the time required to locate and identify an icon. Execution time is the time required 
to respond manually to the stimulus and depends on motor skills and system 


response times. 
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2. Measuring Acquisition Time 
This experiment focused on acquisition time. To measure acquisition 
time, it was necessary to measure total response time, then to subtract out the time 
required for manual response. The subtraction logic technique developed by 
Donders (1869, trans. 1969) and refined by Wickens (1992, p. 335) was used for this 
purpose. That is, a simple baseline reaction-time test (described later) was 
administered to measure subjects’ response times. Average reaction time for each 


subject was subtracted from total response time to obtain acquisition time. 


C. METHOD 


1. Subjects 

Subjects for the experiment consisted of 36 active duty military officers 
enrolled in various master’s degree programs at the Naval Postgraduate School. 
Subjects ranged in age from 27 to 40, with a mean of 31.9 years of age and standard 
deviation of 3.8 years. Four subjects were female and 32 were male; four subjects 
were left-handed and 32 were right-handed. Various levels of computer experience 
were reported by the subjects. The subjects were randomly divided into two groups; 
18 were tested using icons with alphanumeric labels and 18 were tested on the 


unlabeled icons. 


2. Stimuli 
The set of 18 TACAMO icons with and without alphanumeric labels were 


used as test stimuli. In addition to the 18 TACAMO icons, 18 "distractor" icons with 


28 


and without alphanumeric labels were also included to increase search complexity, 
as suggested by Murdock (1982, p. 20). The distractors were developed from 
Modley’s (1976) pictorial symbols to approximate discriminability-index ratings of 
corresponding TACAMO icons. Both the test icons and the distractor icons were 
presented as black images in 0.75-inch white squares on a non-glare light-gray 
background. The stimuli were presented in the form of 324 3X3 randomized icon 
matrices (see Figure 5.2). The complete sets of TACAMO icons and distractor icons 


used for testing are shown in Appendix E.1. 


3. Apparatus 
The experiment was conducted on a Unisys 386 IBM-compatible personal 


computer with an Intel 80387™ math coprocessor, running at 16 mHz. The graphics 


= 





Figure 5.2. Icon Recognition Test Matrix Including TACAMO 
Icons and Distractor Icons 
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were displayed on a 14-inch Unisys color VGA 640X480 monitor. Genus Micro- 
programming’s Proteus™ prototyping software was used to conduct all phases of the 
experiment. Reactions were captured on the keyboard’s numeric keypad keys "1" 


through "9." A time-stamping program written in C was used to capture response 


times from the Unisys system clock (Appendix E.2). 


4. Procedure 
Complete instructions for the experiment were read aloud to each subject 
and biographical data was documented (see Appendices E.3 and E.4, respectively). 
The experiment consisted of five phases: initial icon training, review training, baseline 


reaction testing, additional review training, and icon recognition testing. 


a. Training 
Icon training phases presented a set of 18 TACAMO icons with task 
descriptions. Training was self-paced and identical, except for icon labels, for all 
subjects. Review training phases allowed for self-paced study of the same icons with 


abbreviated task descriptions. 


b. Baseline Reaction Time Testing 
Baseline reaction time testing consisted of a simple search for a black 
block in a 3X3 matrix (see Figure 5.3) and a one-finger response corresponding to 
the location of the stimulus within the matrix. Al] subjects used the index finger of 
the right hand, starting at a common point 5.5 cm directly below the centrally-located 


"D" key on the numeric keypad. The index finger was then returned to the common 
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Figure 5.3. Baseline Reaction Test Matrix 


Starting point before continuing to the next screen for the next test. A total of 27 
reaction times were recorded for each subject. The first nine responses for each 


subject were discarded as warmup trials. 


c. Icon Recognition Tests 

For the icon recognition test, a text-only screen told the subject to 
observe the matrix that would be displayed and to select the icon which best 
represented an abbreviated task description which was provided on the screen (see 
Appendix E.5). As recommended by Murdock (1982, pp. 20-23), subjects were 
instructed to expect that a 50% mix of distractor icons would be randomly included 
in the icon matrices. When the subject was ready to continue, pressing the spacebar 
presented a 3X3 icon matrix. As with the baseline reaction time testing phase, 


subjects used their index fingers to press the key of the numeric keypad which 
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corresponded to the described icon, always starting at and returning to the common 
starting point. Each subject was tested once on every icon in the set. Presentation 
order and matrix position were completely randomized through the use of 324. 


screens of 3X3 matrices. 


5. Performance Measures 
Three performance measures were collected in addition to the 
measurement of baseline reaction time. Response times were recorded for each 
numeric keypad key push. Errors (incorrect key pushes) were recorded, and the 
length of time the subject studied the task descriptions before beginning each test 


was noted. 


D. RESULTS AND DISCUSSION 


1. Subject Variables 
An examination of the intercorrelations for the subject descriptors 
revealed no significant correlations of baseline reaction time and icon identification 
response time to age, sex, or primary hand. Types of computer experience also did 


not correlate to better or worse performance. 


2. Baseline Reaction Test 
An analysis of variance for baseline reaction times showed the subject 
[F35 324 = 19.23, p < 0.001] and the icon’s position in the matrix [F;;., = 2.65, p < 
0.01] to be significant. The interaction between subject and position was not 


significant. 
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The matrix-position effect was compensated for in the main icon 
recognition test by complete randomization of all icons in the nine matrix positions. 
The subject effect was minimized by subtracting each subjects’ average baseline 
reaction time from his or her individual icon recognition times to achieve a 
handicapping effect, essentially lining up subjects’ mean reaction times while not 
disturbing variation (see Appendix E.6). Baseline reaction time correlated weakly 
with total recognition time (r = 0.29). However, total recognition time showed 
strong correlation (r = 0.997) with icon identification time; that is, after baseline 
reaction time was subtracted from the total. The correlation between baseline 
reaction time and icon identification time was very weak (r = 0.08). This indicates 
that subtracting the baseline reaction time from the total response time was a 


satisfactory technique for obtaining icon acquisition and identification times. 


3. Performance Measures 

Response times (with mean baseline reaction times subtracted), error 
rates, and study times were analyzed with both one-way analysis of variance and the 
Kruskal-Wallis test. The-exploratory data analysis (Velleman and Hoaglin, 1981) 
revealed a pattern of outliers, leading to the use of the Kruskal-Wallis nonparametric 
method (Conover, 1980). Significant experimental factors were found to be the 
subject [H;; = 145.03, p < 0.001], icon design [H,, = 134.71, p < 0.001], icon 
position [H, = 31.35, p < 0.001], and labeling [H, = 4.81, p < 0.03}. 

Minitab 7.2™ was used to generate 95% sign confidence intervals for all 


icons, for labeled icons, and for unlabeled icons (see Appendix E.7). Figure 5.4 
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Figure 5.4. 95% Confidence Intervals on TACAMO Icon Identification Times 
(Combined Labeled and Unlabeled with Baseline Reaction Time Subtracted) 


shows the 95% confidence intervals (calculated by the sign method) for identification 
times for the set of 18 TACAMO icons (combined labeled and unlabeled, after 
baseline reaction time was subtracted). All icons were recognized with little response 
time variation with the exception of the Files icon, which fell back on a standard 
desktop metaphor instead of a TACAMO-specific symbol. 

Applicable two-way analysis of variance showed no _ significant 
interactions. Study time was weakly correlated to icon identification time (r = 0.21). 


The discriminability indices were also only weakly correlated (r = 0.22) to 
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identification time for unlabeled icons. A multi-factor analysis of variance model 


could not be calculated due to rank deficiencies resulting from five errors. 


4. __ Error Analysis 

A qualitative analysis of errors showed five incorrect responses (see 
Figure 5.5). Surprisingly, only one of the confusion errors occurred with a subject 
using unlabeled icons; the remaining four errors were committed during subjects’ tests 
with labeled icons. Since the main purpose of the icon label was to provide 
redundant information to avoid confusion, the results warrant a closer investigation 
of the effect of alphanumeric labels on icons. The effect of the labels on the tested 
icon set appears not only to have a detrimental effect on speed, but also to have no 


effect on reducing errors. 


3. Labels—A Closer Look 

For the 643 data points, median acquisition time for icons without labels 
was 1319 ms; for icons with labels, the median time was 1507 ms. Figure 5.6 shows 
the median identification times for =r icon, labeled and unlabeled, in the 
TACAMO icon set. Comparative results show that unlabeled icons yielded quicker 
response times for 15 of the 18 icons. For four of these icons, labeling added over 
400 ms to the median response time. Of interest in Figure 5.6 are the results for the 
Files icon, which show that almost all of the sizable variability shown in Figure 5.4 
comes from the labeled version of the Files icon. Nonparametric sign confidence 


intervals for all icons combined (labeled and unlabeled), for labeled icons, and for 
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Icon Identification Errors 


Type Cause of Error 

Labeled Missed task description 

Unlabeled Confused with Back-up 

Labeled Confused with Preflight 
Labeled Missed task description 

Labeled Confused with Transmit 





Figure 5.5. Errors from Icon Identification Test 


unlabeled icons are contained in Appendix E.7. The Kruskal-Wallis test statistic 
using a comparison of average ranks was calculated as H, = 4.81 with p-value = 
0.029; thus the results are significant at a = 0.05. 

Power was analyzed using StatGraphics 5.0™ statistical software and 
verified with operating characteristic curves as described in Duncan (1986). 
Assuming normal distributions with means of 1645 ms for icons without labels and 
1807 ms for icons with labels, and a pooled standard deviation of 1361 ms, an 
analysis of power shows B = 0.14. Estimating power with the nonparametric 
medians of 1320 ms for icons without labels and 1510 ms for icons with labels lowers 
B to 0.03. The null hypothesis that labels will result in faster response times must 
be rejected at a = 0.05. These results indicate strongly that the alphanumeric 
labeling of well-developed icons does not significantly improve identification times 


or reduce errors. 
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Figure 5.6. Median Identification Times for TACAMO Icons 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


A. SUMMARY CONCLUSIONS 

This study provided important results in two areas. First, more rapid response 
times and lower error rates expected for icons with alphanumeric labels did not 
materialize. In fact, alphanumeric labels were shown to increase response times 
significantly, yet did not result in a lower error rate, for the TACAMO icon set. 
Second, actively involving the targeted user community in the development of icons 
was shown to be a valid method of generating useful icons. The low-cost approach 
used for the evaluation and refinement of icon candidates should be of particular 


value to designers of future military human-computer interfaces. 


B. TACAMO ICONS FOR THE MILSTAR MESSAGE PROCESSING SYSTEM 


1. Icon Designs 
The primary result of this study is a set of icons validated for use in 
TACAMO’s MILSTAR Message Processing System (MMPS). Use of these icons is 
expected to reduce the need for menu commands substantially. The icons should 
provide a faster, more satisfactory human-computer interface since they incorporate 
much more descriptive information using the same (or less) physical display space 
(Gittins, 1986, p. 519). Since the icons were developed with the input of TACAMO 


aircrew personnel, they are expected to be well-suited for their distinct mission, and 
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their meanings should be intuitively obvious to the intended users. It is strongly 
recommended that the Naval Air Warfare Center, Aircraft Division, develop and test 


a prototype graphical interface for TACAMO’s MMPS that will include these icons. 


2. Labeling of Icons 
Labeling the TACAMO icons is not recommended. This study has shown 

that labels do not effectively aid in identification. Furthermore, it is recommended 
that the Department of Defense revise MIL-STD-1472D, Human Engineering Design 
Criteria for Military Systems, Equipment and Facilities, to reflect the findings of this 
study. Namely, Section 5.15.4.8.3 Supplementary verbal labels should be deleted and 
Section 5.15.4.8.2, Iconic menus, be rewritten as follows: 

Iconic interfaces. Graphical system interfaces should incorporate icons to 

represent control options when practical. Where icons are used to represent 

control actions in menus, they shall be designed through production method 

testing and preference surveys of the targeted user. Icons should be 

supplemented with verbal labels only when targeted users are shown to be 

unsuccessful in developing appropriate icons that map to control actions. 
The indiscriminate use of supplementary labels with icons should be discouraged as 


a "band-aid" approach to human factors engineering, an inadequate substitute for 


proper designs. 


C. ICON DEVELOPMENT METHOD 

The icon production method test used during this study provided a quick and 
direct source of reasonable designs for candidate icons. Response to and 
participation in the drawing of candidate icons was positive; the only disappointment 


came from the relatively small sample size of TACAMO aircrew personnel available 
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to assist in the process. With significant benefits achieved at little cost, it 1s 
recommended that the icon production method technique demonstrated here, 
followed by icon refinement through user surveys, be used on a routine basis by 
designers and developers responsible for graphical user interfaces for military 


systems. 


D. LIMITATIONS OF GRAPHICAL INTERFACES 

Icons are not a panacea; they cannot completely replace words in complex 
situations. Every set of new icons should be extensively researched, developed, and 
tested before being introduced into a new application (Marcus, 1991). Based on 
experience gained during this study, several specific cautions about developing icons 
for military systems, such as TACAMO’s MMPS, are offered. 

1. Before any interface format is implemented, a human factors analysis of the 
system must be conducted. A thorough task analysis logically precedes valid 
icon development. 

2. An icon designer should not fixate on a particular type of metaphor (e.g., the 
"office metaphor" prevalent in many of today’s icon interfaces). In essence, this 
shows a lack of originality and an absence of the use of thoughtful design 
methods. 

3. Blanket labeling of icons is an admission of failure by the interface designer. 
The unrestricted addition of alphanumeric labels is a confession that the 
targeted user will not be able to map icons successfully to the task descriptions 
they represent. 

There are two major criticisms leveled against the use of icons. The first is that 


the lack of empirical study and understanding of the cognitive mechanism of icon 


interpretation makes it difficult to recommend design guidelines; thus any particular 
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implementation is arrived at subjectively. The second criticism is that, while icon 
designs can indicate associations between underlying data or functions, this is often 
all that can be achieved. That is, the design can indicate the association, but not the 
actual underlying feature, because of the limits of graphical design techniques. 
(Halasz and Moran, 1982) This study has indicated that both problems can be 
minimized, at least for military systems, by including the intended users in the design 
process. 

These cautions and criticisms warn of the hidden snare of icons—extending 
their use beyond their obvious efficiency as signifiers. Context forms the foundation 
to effective icon development in which a user finds using an interface as obvious as, 
say, ringing a doorbell to enter a house. Icons are quickly becoming the choice of 
many interface designers, since they provide the only interface in which the user does 
not have to remember dozens of facts and processes. Yet the military must employ 
icons judiciously, insisting that each interface be tailored with icons specific to the 
mission at hand. 

Future research on icon development for military applications should focus on 
additional complex issues such as effects of color, long-term memory retention, and 
developing an effective icon-discriminability index. For example, many icons employ 
color while it is clear that the thoughtless use of color can interfere with and 
overload the user’s perceptual process (Murch, 1985). Consideration must be given 


to human perceptual characteristics, physiological factors, cognitive processing, and 
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response capability, for research leading to the optimum design of graphical user 
interfaces for military applications. 

While the scope of this study was limited to the comparison of icons with and 
without labels for a set of icons developed for the TACAMO MMBS, it is hoped that 
the approach taken follows the resourceful path of Jules Henri Poincaré: "avoiding 
the constructing of useless combinations, while constructing the useful combinations 
which are in an infinite minority." Properly developed and designed, icons deliver 


a sophisticated capability, transparent to the operator who must concentrate on the 


mission at hand. 
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TACAMO MMPS PROTOTYPE INTERFACE DESIGN 
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APPENDIX B.1 


TACAMO MMPS INTERFACE GUIDELINES 


1. Goal. The goal of these guidelines is to aid in developing a human-computer 
interface that allows TACAMO aircrew personnel to perform required tasks 
accurately and efficiently. These guidelines are not meant to restate or replace 
existing government and military guidelines, but are designed as a basis for 
developing system-specific operational rules for TACAMO’s MILSTAR Message 
Processing System (MMPS), taking the considerations of Federman (1988) and the 


opinions of fellow ACQs stationed at NPS into account. 


2. Design Principles. The following three principles are recognized by many sources 
(often in varying forms) as underlying a satisfactory human-computer interface 
(Brown, 1986; Card, Moran, and Newell, 1983; Ehrich and Williges, 1986; 
Shneiderman, 1986; Helander, 1988; Laurel, 1990; and Neumann, 1991). The 
TACAMO MMPS prototype was designed with these summary guidelines in mind. 

a. Efficiency. Information should be easy to find. Only information that 
is essential for performing a task should be displayed, minimizing operator effort and 
reducing memory load. Grouping, placement, and sequence characteristics should 
be used to display information in a way compatible with the way operators use the 


information in a given task. The vocabulary of TACAMO aircrew personnel should 
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determine data labels, display titles, menu options, command language, and error 
messages. 

b. Consistency. Consistency in format and operation of the interface 
enhance operator efficiency. Displays can be searched faster for crucial information. 
In particular, location of similar types of information should be consistent from 
screen to screen. General methods of interaction, or habit patterns, should be the 
same regardless of where the operator is in the TACAMO MMPS. Coding, or 
special meanings, such as "blinking red" for alarms, should be applied consistently to 
ensure that meanings are clearly interpreted. 

C. Feedback. Every operator action should invoke a noticeable response 
from the computer. Specifically, when the operator has made an error or performed 
an action that the TACAMO MMPS does not understand, the TACAMO MMPS 
should provide a message of explanation and method of recovery. When an operator 
performs an action that has potential destructive effects, the TACAMO MMPS 
should require confirmation before processing the request. When the TACAMO 
MMPS is busy performing a lengthy process, it should notify the operator through 
a system message or display graphic. When it is appropriate for an operator to 
select objects of the display to manipulate, the system should indicate that an object 
has been selected (by image reversal highlighting or color coding) so the operator 


knows on what object the selected action will be directed. 
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3. General Screen Characteristics. Figure B-1.1 shows the TACAMO MMPS 
prototype display format. A graphical interface with pull-down menus and an iconic 
interface should make the TACAMO MMPS simpler and easier to use than the 
current TACAMO Message Processing System (TMPS) (Hildebrand, 1991). 
Constant monitoring of all radio circuits was deemed a high priority, as was 
establishing a visual message alarm system, distinguishing the message editing screen, 
and providing an extensive online help capability. 

a. Pull-down Menus. Pull-down menus are displayed as a horizontal list 
of options at the top of the screen. Proposed MMPS menu structure is listed in 
Appendix B.2. The menus were ordered by grouping similar tasks as described by 
Harpster (1987) and McDonald, Dayton, and McDonald (1988). Surprisingly, it 
appears that the names given to commands have little effect on menu selection times 
(Smelcer and Walker, 1990). Operator search time is significantly shorter for the 
vertical list format (Backs, Walrath, and Hancock, 1987). The menu commands 
should be mixed case with double spacing between menu commands (Williams, 
1988). 

b. Iconic Interface. An appropriate icon may be easier to recognize and 
comprehend than any amount of text (Brown, 1987, p. 91). By presenting user 
command and system information in the form of icons, the TACAMO MMPS can 
capitalize on the new capabilities of graphic displays, reducing both time and effort 
of learning and facilitating performance while reducing errors (Lodding, 1983, p. 11). 


Ordering of the icons corresponds to ordering of the menu commands. While not 
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all menu commands have a corresponding icon, all icons have a corresponding menu 
command. 

With the established superiority of icons over alphanumerics for many 
applications (Steiner and Camacho, 1989), an iconic interface is strongly endorsed 
as an alternative interface for the TACAMO MMPS. While icon interfaces are 
aimed primarily at the novice (Blankenberger and Hahn, 1991), studies have shown 
that icons produce faster search and selection times (Camacho, Steiner, and Berson, 
1990) and in some cases they are even preferred by experienced operators (Guastello 
and Traut, 1989). 

e: Monitoring Radio Circuits. The bar at the bottom of Figure B-1.1 will 
be used to monitor incoming radio traffic. Current system capabilities do not allow 
the operator to monitor all of the circuits received aboard TACAMO at the TMPS 
operator position. Each circuit identifier would consist of a small blue button with 
an abbreviated circuit label in dark gray. When a circuit is actively monitored, the 
label would turn bright white, indicating normal functions. A circuit under repair or 
malfunctioning would be indicated by a light gray button. Incoming traffic would 
turn the labels yellow. Red labels on yellow buttons with a "3-D popped circuit 
breaker" effect should be used for Flash or higher precedence message traffic. By 
clicking on any button, the operator should be able to directly access the desired 
circuit. The "3-D popped circuit breaker" effect would reset itself after the operator 
had inspected the incoming message traffic to ensure Flash or higher message traffic 


was promptly handled. 
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d. Message Alarm System. A visual alarm system for incoming messages 
would eliminate the need for the current aural alarm that is rarely used due to its 
ineffectiveness (Federmann, 1988, p. 2-3). Figure B-1.1 shows a half-circle fan in the 
upper right-hand corner of the screen. This half-circle would serve as a dual-purpose 
gauge, normally used to indicate space remaining on the fixed-disk drive, but turning 
into a visual alarm for incoming messages. The half-circle would be light gray with 
blue right-to-left shading when indicating the amount of space remaining on the fixed 
disk. The blue fill would turn to amber, to warn when the disk capacity exceeds a 
specified percentage. The half-circle would become a spinning yellow-red fan when 
a Flash or higher precedence message arrives aboard the aircraft. 

é. Editing Screen. An extremely serious error occurs when the operator 
is confused about whether the present screen displays a message that has been 
received or a message that is being edited. A clear distinction between the two types 
of screens is definitely needed, as several TMPS operators have registered complaints 
with the split screen used in current equipment (Federman, 1988, p. 2-2). The 
message monitoring screen should have a light yellow background with text displayed 
in a blue, sans serif font similar to that used with older teletype machines. The 
editing screen should be paper white with a black serif font for high legibility 
(Oborne and Holton, 1988). For higher resolution screens, light-colored type on 


dark backgrounds produces much slower reading times (Shneiderman, 1987, p. 360). 
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If additional distinction is needed between the two display formats, simulated 
perforation tracks could displayed on the right and left edges of the message 
monitoring screen. Operators should have the option of toggling quickly between 
screens and also the option of viewing more than one screen at a time, each in a 
separate window. For message editing purposes, a temporary "paste-up" buffer space 
should be made available to cut, store, and paste sections of different messages. 

ii Screen Colors. In this proposal, menu bars are dark blue with white 
lettering. Bright yellow is used for the single-letter "speed-keys" within each menu 
command. Reverse video (white background with blue letters) should be used to 
highlight pending menu selections. Icons are drawn with dark blue details on a white 
background for discriminability. Similar to the menus, reverse video (dark blue 
background with white details) indicates a pending icon selection. Unusable space 


should be designated with a separate color, such as light blue. 
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g. Online Help and Training. Online help should be made available so the 
operator can access information about features of the software and can obtain aid 
in problem recovery. Easily accessed information helps the user cope, facilitates 
browsing, and results in the rapid learning of new features or advanced techniques 
(Heckel, 1991, p. 57). Context-sensitive help is especially valuable in giving operators 
quick answers to specific problems. Prototype testing and evaluation should provide 
detailed documentation of problem areas in need of context-sensitive help. 

The TACAMO MMPS should be used extensively for training 
purposes. An ideal TACAMO aircraft would have more than one MMPS, with one 
terminal available at all times for training or as a backup to the main terminal. 
CD-ROM-based training and publications storage would fully exploit the capabilities 
of a well-planned TACAMO MMPS, reducing aircraft weight and enhancing 


survivability. 


4. Interaction Methods. 

a. Menu Selection. For the process of selecting an item from a menu, 
cursor keys, a trackball, scrolling bars, and "speed-keys" should be implemented. 
Cursor keys operate pull-down menus and can even control icon menus, if desired. 
Trackballs have been shown to be an effective method of interaction, providing direct 
access by point and click. The trackball is preferred over the mouse for applications 


involving high stress and limited space (Shneiderman, 1987, p. 242). 
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The usual wisdom is that pointing devices, such as a trackball, mouse, 
or joystick, are faster than keyboard controls, however, studies have shown that for 
tasks that mix typing and pointing, keyboard input is often preferred and results in 
less muscular strain (Shneiderman, 1987, p. 247). Scrolling bars should be provided 
to allow operators to screen message traffic quickly. The use of "speed keys," single 
keypresses that access a command directly (usually indicated via a single highlighted 
letter on a menu command), offers advantages common to both cursor keys and the 
trackball. Eliminating the requirement of pressing the Enter key translates to a more 
direct method of entry, often preferred by experienced users. 

Menus should always be available for repeated use if desired. For 
example, if an operator desires to switch radio frequencies for multiple circuits and 
for each radio must access the frequency control menu that is three levels down and 
three levels up, the operator will quickly become annoyed. Operators should always 
have instant access to the top level menu if desired. The most important aspect of 
menu selection control is providing all reasonable methods of control so that the 
individual operator can develop a style of operation that he or she is most 
comfortable with. 

b. Data Entry. For data entry, direct entries are significantly faster and 
preferred by system operators (Gould, et al., 1988). This finding should not 
discourage the use of pre-filled data fields for common entries, such date-time-group 


entries, latitude-longitude entries, and pre-formatted message forms. 
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C. Innovative Graphical Displays. Electronic Post-[t™ notes, scheduling 
aids, and other real-world conveniences should be made available to the operator. 
Open communication lines between the prototype designer and the actual TACAMO 
operators will ensure that such conveniences will be provided in Version 1.0 and - 


as a later add-on. 


5. Development of a Prototype. Full development of a TMPS MMPS prototype 
should directly involve the TACAMO community at all stages of testing. Early 
evaluation and thorough iterations will produce a satisfying end product. The use 
of PC-based design and prototyping tools will result in a better design process, 
allowing the interface designers to involve TACAMO aircrew personnel actively, 
resulting in a significantly better user interface design (Koster and Wilkinson, 1988, 
p. 360). 

Melkus and Torres (1988) provide excellent guidelines for using prototyping 
with a minimum of problems. For a more thorough treatment on prototyping, 
Wilson and Rosenberg (1988) describe available tools and evaluation techniques to 
aid in prototyping from the "tangible speculation" stage to the final testing and 


analysis. 
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APPENDIX B.2 


MMPS MENU STRUCTURE 


MESSAGES=—=5 
Transmit => 
Select message and parameters 
Monitor => 
EHF or MILSTAR 


UHF 
VLF 
HF DATA 
Queue of outgoing messages 
EDIT= 
Create 
Retrieve 
Save 
Print 
Punch tape 
Spellcheck 
Block 


Display message files—= 
Flash messages 
General catalog 
Standard message forms 


STATUS => 
Comm Central==> 
Power 
AN-USC/13 
Reel assembly 
Aircraft systems} 
Status 
PMS 
TACAMO bases 
WWABNCP== 
Basing 


Airborne schedule 


a5 


EQUIPMENT=5 
Pre-Flight tests 
Configure Comm Central—> 
Set circuits 


Set alarms 
Verify FTS 
Back-up mission data 
TRAINING== 
PQS training 
Crew training 
Mission goals 
Squadron goals 
LIBRARY => 
Abbreviations & acronyms 
Dictionary 
Thesaurus 
Professional= 
COG 
TOP 
OPORDERS 
NATOPS 
Maintenance 
HELP=—} 
Help 
Index 
Version 
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10. 


11. 


12. 


13: 


14. 


15. 


16. 


i? 


iS. 


ro. 


20. 


APPENDIX B.3 


INITIAL TASK DESCRIPTIONS FOR MMPS ICONS 


Preflight tests and checklists 

Display message catalog (files) 

TACAMO aircrew basing information 

Transmit a message 

Retrieve an existing message 

Print a paper copy of a message 

General help index for MMPS software 

Aircraft systems readiness status information 
Configure Comm Central equipment 

Spell check a message 

Block and cut a section of a message 

Create a message 

Operational, NATOPS, and maintenance publications 
PQS syllabus training, programmed texts 

Thesaurus 

Cut a teletype tape of a message 

Display the outgoing message queue 

Backup mission data from the hard disk to floppy diskette 
Calendar of scheduled events 


Save a message 
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APPENDIX C 


ICON PRODUCTION METHOD TEST 
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APPENDIX C.1 


TACAMO ICON PRODUCTION QUESTIONNAIRE 


Name (optional): 


Rank/Rate: Age: Sex; M F 





TACAMO experience: 

Crew position(s) 

Aircraft flown 

Time in TACAMO community yTS. mos. 
Education: 

High School/GED 

Some College Courses 

Associate’s Degree 

College Degree 

Postgraduate 
Computer experience (circle as many as applicable): 

IBM PCs Macintosh Mainframe terminals 


Amiga Commodore Other 


Subject number 


Shy 


MENU STRUCTURE SURVEY 


For each question, please darken the circle nearest the level of importance 
you would place on each task for inclusion on the Milstar Message Processing 


System. 


0 OD oO ee 


~ 


Preflight tests and checklists. 

Display message catalog. 

TACAMO aircrew basing information. 

TACAMO abbreviations and acronyms dictionary. 
Transmit a message. 

Chain of command organization chart. 

Retrieve an existing message. 

Print a paper copy of a message. 

General help index for MMPS software. 

Aircraft systems readiness status information. 
Standard dictionary. 

Configure Comm Central equipment. 

Spell check a message. 

Squadron goals. 

Mission commander goals. 

Comm Central readiness status information. 

Crew deployment training goals. 

Monitor a radio circuit. 

Block and cut a section of a message. 

Create a message. 

Operational and maintenance publications, NATOPS. 
PQS syllabus training, programmed texts. 

Thesaurus. 

Cut a teletype tape of a message. 

WWABNCP basing and deployment status information. 
Version number for MMPS software. 

Display the outgoing message queue. 

Backup mission data from the hard disk to floppy diskette. 
Calendar of scheduled events. 

Save a message. 
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MENU STRUCTURE SURVEY 


Very 
Important 


Not As More 
Important 


Not 
Important 


Important 


Important 


AN 
dic 


om st 
ee 


JL) 


Zr 


Pa AN 
Za. 
23. 


Zor 


Pag 
28. 
PS 





30: 


Subject number: 
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ICON PRODUCTION TASK DESCRIPTIONS 


The following task descriptions were presented verbally and on an overhead 
projector slide in this random order: 


0 ON DY SS See 


Oe 
oN ON DN UF YW NY FF SO 


Preflight tests and checklists. 

Display message catalog (files). 

Create a message. 

Operational and maintenance publications, NATOPS. 
General help index for MMPS software. 
Aircraft systems readiness status information. 
Block and cut a section of a message. 
TACAMO aircrew basing information. 
Retrieve an existing message. 

Print a paper copy of a message. 

Display the outgoing message queue. 
Transmit a message. 

PQS syllabus training, programmed texts. 
Thesaurus. 

Cut a teletype tape of a message. 

Configure Comm Central equipment. 

Spell check a message. 

Save a message. 

Calendar of scheduled events. 


Backup mission data from the hard disk to floppy diskette. 
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OOOO 
PILI LI LIL 


PILI LI LIL 
eet | Lt 
PILI LI LIL 
PIL IL LIL 


APPENDIX C.2 


ICON PRODUCTION INSTRUCTIONS 


I am interested in developing icons for use in a graphical interface for the 
next-generation MILSTAR Message Processing System (MMPS). Go ahead and fill 
out the background information on the first page of your questionnaire before we 
begin. I have focused on the use of icons in the MMPS because icons seem to be 
recognized more rapidly and accurately than command lines that are currently used 
in the TACAMO Message Processing System (TMPS). Standard symbols and 
pictograms are currently in use in automobiles, public transit systems, and highway 
signs. 

The key to developing strong TACAMO icons is your participation here 
today. The method I’m using today is refered to as the icon production method 
which has been used to succesfully develop Army vehicle controls, automobile labels, 
and more recently Army battlefield symbology. 

So, what makes a good icon? A 
strong pictorial representation, for 
example (draw figure on board): 

This windshield wiper is an excellent 


example of an icon that provides a 





strong image with an image of limited 
complexity. Today we’re going to try 


Control Symbol for Automobile Wiper 
and create symbols for TACAMO icons 
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with similar strengths. 
Basically, a good icon will be: 
1. Meaningful, 
IE: Simple, yet bold, 


3. Different looking from those that already exist—especially those 
that exist on your nieghbor’s paper. 


Don’t be afraid to explain your artwork. This is not a test of your artistic talent. Try 
to draw each icon as large as the space allows. For each task description, please 
draw a picture in the top box and an alphanumeric label in the bottom box provided. 
You will be given two minutes to draw each icon. 

Do you have any questions? 


Thank you for your time and cooperation. 
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APPENDIX C.3 


ICON PRODUCTION METHOD DRAWINGS 
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LILES 
Le 


Create a message 


Operational and maintenance publications, NATOPS 
PuOS 


Display message catalog 


Preflight tests and checklists 


Block and cut a section of a message 


General help index for MMPS software 


ABOAE 


=| [1 [= 
HROAB RE 


Ee 
HUBLI 


eel 
A 
ee 
——~ 
SS 
——— 


craft systems readiness status information 


4 OSE 
Bl 
ae 
bl & 
1h & 
AO 
7] 
efile 


TACAMO aircrew basing informatio 


OO @ae 
IOS E OAR 
Oa CIN 
EO §He 
IOS OMe 
Mees lo@e 
@ BH ae 


“AE Se 





Tele) OC 
aioe e 
NGO Bee 
TAS) Bele 
OO0 OOF 
OR Oe & 
al‘ ele 
ADD Bee 
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Cut a 7 tape of a message 
Configure Comm Central equipment 


PQS syllabus training, programmed texts 


Calendar of scheduled events 


}LI LJ ed el bel EJ 


Spell check a message 


LJ 6) [=] | 
=) |-}L] 4 le} L] lal [eB 


A 


Ka 


Backup mission data from the hard disk to floppy diskette 





ie 


Le! fe 
(2 Ls) 
I) || | 
EG 


APPENDIX C.4 


LABELS FROM ICON PRODUCTION METHOD 


Task description Proposed labels (responses) 


Preflight tests and checklists. PREFLT (5) 
PREFL 
PREFLIGHT 
PFCL 
8) ig 
LISTS 


Display message catalog (files). DMC (4) 
CAT (2) 
FILES(2) 
CATALOG 
MESSAGE 
MSG CATALOG 


Create a message. CREATE (4) 
CREATE MSG (2) 
NEW (2) 
NEW MSG (2) 
MSG 


Operational and maintenance PUBS (6) 
publications, NATOPS. NATOPS (4) 


General help index for MMPS software. HELP (6) 
HLP 
GEN HELP 
MMPS 
INDEX 


Aircraft systems readiness status ACFT STATUS (3) 
information. ACFT 
A/C STATUS 
SYSSTAT 
STATS 
STS 
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10. 


11. 


12. 


Nx 


Block and cut a section of a message. 


TACAMO aircrew basing information. 


Retrieve an existing message. 


Print a paper copy of a message. 


Display the outgoing message queue. 


Transmit a message. 


PQS syllabus training, programmed texts. 
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CUT (7) 

BLOCK (5) 

BLK 

PASTE 
INPUT/OUTPUT 
MSG 


BASING (5) 
BASES (2) 
TACBASING 
TACBAS 
BSE 


RETRIEVE (3) 
RETR (2) 
RTMSG 
RTVMSG 

OLD 

RECALL 

GET 


PRINT (4) 
PRT (3) 
PRINTMSG 
PRTMSG 
PTMSG 


QUEUE (4) 
MSGQUE (2) 
DSPYQUE 
DISPLAY 
OUTGOING 
OUTMSG 


XMT (4) 
XMIT (3) 
TMT 


TRANSMIT 
TRANXMSG 


PQS (8) 
TRNG (2) 
TRAING 


14. 


i. 


16. 


Wee 


18. 


i. 


20. 


Thesaurus. 


Cut a teletype tape of a message. 


Configure Comm Central equipment. 


Spell check a message. 


Save a message. 


Calendar of scheduled events. 


Backup mission data from the hard 
disk to floppy diskette. 
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THESAURUS (5) 
THSRS 

TSURS 

THES 

THS 


TTYTAPE (3) 
TAPE (2) 


TTY 
CUTTTYTAPE 
CUTTAPE 
REPERF 


CONFIG (4) 
CONFIGURE (2) 
COMMCENT STATUS 
REDSTAT 

COMM 

CEC 


SPELLCHECK (5) 
SPELL (4) 
SPELLCHK 


SAVE (6) 
SAVEMSG (3) 
SMSG 


CALENDAR (6) 
CAL) 
CALDR 


BACKUP (3) 
FLOPPY (2) 
BACKMSG 
BKP 

DISK 

COPY 


APPENDIX C.5 


PICTURE ELEMENTS FROM ICON PRODUCTION METHOD 


Task description Picture elements (responses) 


ie Preflight tests and checklists. 
Checkmark (5) 
Pocket checklist (2) 
Aircraft with checkmark 
Check on paper 


je Display message catalog (files). 
Paper files (5) 
Paper with arrow (2) 
Book (2) 
Rolodex file 


oF Create a message. 
Paper (2) 
Person with paper 
Person with question mark 
Pen, ink, and paper 
Pen and ink 
Open book 


4. Operational and maintenance publications, NATOPS. 
Books (3) 
Paper with zzz’s 


2. General help index for MMPS software. 
Question mark (2) 
Help cross (2) 
Person with question mark 
S-O-S in morse code 


6. Aircraft systems readiness status information. 


Aircraft outline (5) 
Aircraft with up/down arrows (2) 
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10. 


JU, 


1 


Block and cut a section of a message. 


Scissors with message (3) 
Scissors (2) 

Message with highlights (2) 
Block 


TACAMO aircrew basing information. 


Map of the United States (3) 
Motel (3) 

Aircraft with message 

Bed 

Telephone 

Stopsign 


Retrieve an existing message. 


Computer, arrow to paper (2) 
Dog with bone (2) 

Computer, arrow to monitor 
Monitor with "R" 

Hand grabbing 


Print a paper copy of a message. 


Printer with paper output (6) 
Stack of paper 

Piece of paper 

Arrow to paper 


Display the outgoing message queue. 


Transmit a message. 


Letter "Q" (2) 

Aircraft, lightning bolt, question mark 
Satellite dish 

Message 


Lightning bolts (3) 

Satellite dish with lightning bolts (2) 
Circuit symbols (2) 

Aircraft with lightning bolt 

Drogue with arrows 

Drogue 


fad) 


ie 


14. 


i: 


16. 


We 


18. 


SE 


20. 


PQS syllabus training, programmed texts. 
"PQS" in box 
Clipboard 


Thesaurus. 
Book with "T" or thesaurus 
Book with question mark 


Cut a teletype tape of a message. 
Teletype tape and scissors (2) 
Teletype tape (3) 
Reperforator (2) 
Roll of teletype tape 


Configure Comm Central equipment. 
Comm central likeness (4) 
Patch panel 
Oldtime radio 
Face with "CONFIG" bubble 
Aircraft with up/down bubble 


Spell check a message. 
"SPELL" with checkmark (3) 
Spelling bee 
Book with checkmark 
CAT/KAT misspelling 


Save a message. 
Monitor, arrow to computer (2) 
Message, arrow to computer 
Diskette 
Life ring 
Baseball diamond 


Calendar of scheduled events. 
Calendar month (10) 


Backup mission data from the hard disk to floppy diskette. 


Diskette (5) 
Computer, arrow to diskette 
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APPENDIX D 


ICON CANDIDATES 


ve, 


APPENDIX D.1 


INITIAL TACAMO CANDIDATE ICONS 


TRANSMIT QUEVE STATUS 


1. Transmit a message 2. Queue of outgoing messages 





CREATE MESSAGE DISPLAY CATALOG 





3. Create a message 4. Display message files 





5. Print a message 6. Punch a teletype tape 
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SPELL CHECK 


7. Save and retrieve messages 8. Spellcheck a message 





BLOCK CUT STATUS 


9. Cut a block of a message 10. Status of aircraft equipment 





PREFLIGHT TESTS 


11. Configure Comm Central 12. Preflight checklists 
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ABBREVS. & ACRONYMS 


A pic | - 
pif B NAV 
Cc AB 


13. Backup messages to diskette 14. Abbreviations and acronyms 





PUBLICATIONS MAINT. PUBS 


15. Publications library 16. Maintenance manuals 








17. Training and qualifications 18. Help and information 
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APPENDIX D.2 


TACAMO ICON SET REVISIONS 


1. Transmit a message. Original ideas were abundant for this icon, from a picture 
of the drogue at the end of the transmitting wire to a telegram symbol. The arrow 
bolting downward from the silhouette of a TACAMO E-6A aircraft was chosen to 
signify the transmission of a message from the TACAMO aircraft. The arrow was 
used as a common attribute in icons used to depict message movement in this icon 
set. 


2. Queue of outgoing messages. A capital "Q" was the clearest representation 
available to represent this abstract icon, which will be used to check the flow of 
outgoing messages. Various fonts were considered, but a DrawPerfect’s WP Century 
Schoolbook font gave the "Q" its distinctive tail. 


3. Create a message. The scroll and quill design was borrowed from the Navy’s 
enlisted journalist’s rating badge. Pens, inks, and even word processor’s were 
considered, but the rating badge drew high approval marks from Navy personnel. 


4. Display message files. The old TMPS terminology for this icon would be "display 
message catalog." Drawings and ideas were rather weak on this task description, so 
an office metaphor of a group of file folders was chosen over a picture of a file 
cabinet. 


5. Print a message. This icon show a printer with paper feeding out of the top of 
it. This version may have drawn high support because it closely resembles the 
teletype printer currently used aboard the TACAMO aircraft. 


6. Punch a teletype tape. A spool of teletype tape was drawn for this icon. It was 
reversed on the y-axis to prevent confusion with the question mark used for the help 
icon. 


7. Save and retrieve message. These icons also employ arrows. Save signifying a 
message being stored in a computer’s fixed hard drive. Retrieve signifying a message 
being retrieved from computer memory. Arrow directions were modified between 
the save, retrieve, and backup icons to reduce confusion from similar arrow types. 


8. Spellcheck a message. This task description was described by most as the 


toughest to represent pictorially. The checkmark seemed natural, but glasses were 
added to represent additional focus. 
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9. Cut a block of a message. The scissors cutting into a block depicts a common 
text editing function. 


10. Status of aircraft/equipment. The thumbs up and thumbs down with question 
marks for uncertainty was chosen for this icon. The thumbs were chosen to replace 
arrows to lessen confusion with arrows representing message movement. 


11. Configure Comm Central. This icon shows three plugged-in patch panel 
connections, similar to communication patches made aboard the TACAMO aircraft. 


12. Preflight checklists. The aviator’s three-ring checklist with a large check through 
it was a natural for this icon. 


13. Backup message to diskette. The arrow shows a message being stored to a 3.5" 
diskette, the size of diskette anticipated for use with the improved MMPS. 


14. Abbreviations and acronyms. A book with "ABC" and an abbreviation for the 
Dictionary of Naval Abbreviations was considered for this icon which was later 
dropped from the study. 


15. Publications library. A set of books is shown for this pictorially-based icon 
drawn from Modley’s (1976) book of pictorial symbols. 


16. Maintenance manuals. A separate icon for use by the flight engineers is 
designed to give direct access to the aircraft’s maintenance and equipment manuals. 


The wrench inside a large book neatly combines both ideas. 


17. Training and qualifications. An open book signifying studying is laid across a 
set of aircrew wings for this icon. 


18. Help and Information. The question mark, a universal symbol for help and 
information, was a clear selection for this icon. 
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TRANSMIT 
SAVE 
RETRIEVE 
BACKUP 
PREFLIGHT 
QUEUE 

eid ee: 
TAPE 
CONFIGURE 
CALENDAR 
BASES 
CREATE 
PRINT 
PUBS 
MAINT 
TRAINING 
CUT BLOCK 
SPELLCHECK 
STATUS 


c = 20 


Icon 

HELP 
TRANSMIT 
SAVE 
RETRIEVE 
BACKUP 
PREFLIGHT 
QUEUE 
FILES 
TAPE 
CONFIGURE 
CREATE 
PRINT 
PUBS 
MAINT 
TRAINING 
CUT BLOCK 
SPELLCHECK 
STATUS 


c = 18 


APPENDIX D.3 


DISCRIMINABILITY INDICES 


X1 X2 25-2 Z OI 

18 0 23 2 2.25 
19 16 20 5 2.30 
I. 16 20 5 2.30 
19 16 20 5 2.30 
19 16 20 5 2.30 
18 ee 20 5S) 2.30 
19 0 24 1 2./0 
19 0 24 1 2.70 
19 0 24 1 2./0 
19 0 24 1 2.70 
ile 0 24 1 2.70 
19 0 24 1 2.70 
19 17 21 + ya as. 
19 ee 21 4 2.75 
19 17 21 4 2.75 
19 17 21 4 2.75 
19 17 21 4 2.75 
19 18 22 3 3.20 
19 18 22 3 3.20 
19 18 22 3 3.20 


where DI = 0.07[X1+X2+(25-Z)] - 0.312. 


REVISED DISCRIMINABILITY INDICES 


X1 X2 23-2 Z Ol 

16 0 21 2 1.83 
A 14 18 5 1.88 
le 14 18 5 1.88 
17 14 18 5 1.88 
17 14 18 5 1.88 
16 15 18 5 1.88 
70 0 22 1 2.28 
li 0 22 1 2.28 
ly 0 22 1 2.28 
17 0 22 1 2.28 
17 15 19 4 2.33 
17 is 19 4 2.33 
17 tS 19 4 2.33 
17 1e 19 4 2.33 
le 1s 19 4 2.33 
17 16 20 3 2.78 
17 16 20 3 2.78 
1, 16 20 3 2.78 


where DI = 0.07[X1+X2+(23-Z)] - 0.31Z. 


85 


APPENDIX D.4 


CONFUSIBILITY MATRIX 


Response 
(subject selection) 
Actual 
Stimulus 1 2 3 4 5 6 7 8 9 10 11 #12 13 14 15 16 17 #18 19 20 Other 


poo) eS SS Se 
aye tes} TE Tr DL ULL eS eS eee 
sy ot pool tt Se 
at tt pest tT tT Et eee 
57 st | iy te] | Tt) ol Ss ee ee 
6] of St sfvol [Sle Te | fet] le ine ST 
7T t tot st | fest a 
sy tt | tT to) | ee 
ot tT yt tt tt fool ee 
opt tt tt eo eee 
me oT tf st oy fT st | ett | Oe 
ae | Et tT Tt oo) ee 
at Tt tt tT tT tt Tt st a 
map ot tt Tt | TT | | tt ot ee 
15] ) LAS ee DP co 
eT tec 
mot Tt | tT eel 
me Tt Lt el es 
aT tT Pt tt OE eee 
zof | tT | | dT dE Tt | a 


This matrix represents the actual stimuli as rows and the responses of the 
subjects as columns. Each number represents a percentage of the subjects’ 
responses. Entries along the main diagonal represent recognition and off-diagonal 
entries represent confusion. 
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APPENDIX E 


ICON DESIGN FORMAT EXPERIMENT 
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APPENDIX E.1 


TACAMO AND DISTRACTOR ICONS FOR FINAL TESTING 





3/Q 
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mee 
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APPENDIX E.2 


TIMESTAMP PROGRAM 
STAMP .C 


#include <dos.h> 
#include <stdio.h> 


void main(int argc, char **argv) 


struct time t,s; 
ES aiior 


if((fp = fopen("datafile", "at"))== NULL){ 
printf("FILE did not open correctly. Check memory management"); 
exit(1); 


} 

if(argce != 2){ 
printf("Wrong number of parameters were passed"); 
exit(1); 

} 


gettime(&t); 

fprintf(fp, "%02d:%02d.%02d\t%s\n", 
t.ti_min, t.ti_sec, t.ti_hund, 
argv[1}); 


fclose(fp); 


Disclaimer: The reader is cautioned that this program may not have been exercised 
for all cases of interest. While every effort has been made, within the time available, 
to ensure that it is free of computational and logic errors, it cannot be considered 
validated. Any application of this program without additional verification is at the 
risk of the user. 
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APPENDIX E.3 


INSTRUCTIONS FOR ICON DESIGN FORMAT EXPERIMENT 


Background. The Naval Air Development Center is supporting thesis research from 
the Naval Postgraduate School for the development of future TACAMO 
communications upgrades. This experiment is the final testing stage for an icon set 
developed for use aboard the TACAMO airborne communications platform. The 
experiment consists of five phases conducted on a personal computer: 


Initial Icon Training 
Review Training 

Baseline Testing 

Additional Review Training 
Icon Recognition Testing 


Wie Une 


Your participation will indicate how well each of the developed icons is associated 
with its intended purpose. 


Purpose. This experiment will be used to measure the effectiveness of the icon set 
to increase performance while reducing errors. 


Icon Training. The initial icon training phase wil present eighteen icons with task 
descriptions. All training is self-paced and typically advances with the pressing of the 
spacebar. Review training presents the icons again with abbreviated task 
descriptions. 


Baseline Testing. Baseline testing uses the numeric keypad keys one through nine 
with a simple search for a black block in the 3X3 matrix. The index finger/forefinger 
of the right hand will start at a specified point beneath the numeric keypad and 
should return to that point before continuing. 


Primary Task. In the main icon recognition test, a text-only screen asks you to select 
the icon which best represents an abbreviated task description. When you are ready 
to continue, pressing the spacebar will present a 3X3 icon matrix. Press the key of 
the numeric keypad which corresponds to the described icon and return your finger 
to the starting point. In addition to the eighteen TACAMO icons, eighteen arbitrary 
icons have been randomly included in the search matrices. You are asked to work 
as accurately and quickly as possible. If you have any questions, please ask them at 
this time. 
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APPENDIX E.4 


TACAMO ICON RECOGNITION EXPERIMENT QUESTIONNAIRE 


Age: Sex: M F Primary hand: L R 


Education: 
High Schoo/(GED 
Some College Courses 
Associate’s Degree 
College Degree 


Postgraduate 


Computer experience (circle as many as applicable): 


IBM PCs Macintosh Mainframe terminals 


Amiga Commodore Other 
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10. 


11. 


12s 


13. 


14. 


[By 


16. 


NE, 


18. 


APPENDIX E.5 


ICON TASK DESCRIPTIONS 


Transmit a message. 

Queue of outgoing messages. 
Display message files. 

Create a message. 

Save message to computer storage. 
Retrieve message to display monitor. 
Print a message. 

Punch a teletype tape. 

Cut a block of a message. 
Spellcheck a message 

Back up a message to diskette. 
Preflight checklists. 

Status of aircraft/equipment. 
Configure patch panel. 
Publications library. 

Maintenance manual. 

Training and qualifications. 


Help. 
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APPENDIX E.6 


EFFECTS OF BASELINE SUBTRACTION 


E.6.A. BASELINE REACTION TIME DATA BEFORE SUBTRACTION 
OF AVERAGE REACTION TIME 


oho be Sic ao oats Sack oe a a foo aeGae 
4.00 4.20 4.40 4.60 4.80 5.00 


Each dot represents 7 data points. 


E.6.B. BASELINE REACTION TIME DATA AFTER SUBTRACTION 
OF AVERAGE REACTION TIME 


sees e#eeseseeeseeeseseseesesee 
*®* es seesseseeeseeeeeseseseeeseseeese 


evo ese @ @ oe 8 @#e8e8e 88? @ 8 8 8 8 @ 8 @ 8&8 @ @ 
ss seseseeseesese @eseeeseseeseeeseseseeses8e8 8 8 8 


-0.15 0.00 0.15 0.30 0.45 0.60 


Each dot represents 3 data points. 
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E.6.C. 


SUBJ 
SUBJ 
SUBJ 


SUBJ 


SUBJ 


SUBJ 
SUBJ 


SUBJ 


SUBJ 


SUBJ 
10 


SUBJ 
13 


SUBJ 
14 


SUBJ 
15 
SUBJ 
16 


SUBJ 
17 


SUBJ 
18 


DOTPLOTS OF SUBJECTS’ 


BASELINE REACTION TIME 


BEFORE SUBTRACTING AVERAGE REACTION TIME 


he terre ee--- +o eee ee ee- +--------- tee --- eee teoee- Sec . 
eter c eee eee teeccccce- i teecc cece te erccccee teceee Sec 
ee toccee eee tec cece eee to cc cee eee torr cee eee +eo-o--- Sec 
ot--------- i ter cece teecrecece- terre ee ce- t+-ree- Sec 
ote ccc cece te -- eee i 5 a i tee--e Sec 
ie 5 tere cere +--------- toe c cee ee teeree Sec 
ot- eee ---- sc +-- eee eee i to --- eee +----- Sec 
ot-- eee eee 5 +--------- t--- eee eee 9 teres sec 
eter ere cece terre cceee- a i ii 5 +----- Sec 
ote eee eee i se t------ eee i te--ee Sec 
ete ee eee eee +e -------- to-------- +--------- +o -- eee eee +----- Sec 
-t--------- i tee - eee i t--------- +----- Sec 
-+--------- t--------- +--------- +--------- +--------- +----- Sec 
et--------- +--------- too -- eee +--------- +eo-o- ee eee +----- Sec 
ot--------- +------- ee toe -e-- eee +eo-c-c eee eee tore eee eee +----- Sec 
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E.6.D. DOTPLOTS OF SUBJECTS’ BASELINE REACTION TIME 
AFTER SUBTRACTING AVERAGE REACTIOM TIME 
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APPENDIX E.7 


95% SIGN CONFIDENCE INTERVALS FOR IDENTIFICATION TIME 


i.7.A. LABELED AND UNLABELED TACAMO ICONS COMBINED 
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E.7.B. UNLABELED TACAMO ICONS 
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E7.C. LABELED TACAMO ICONS 
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